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What is The Nation's Report Card? 



The Nations Report Card, the National Assessment of 
Educational Progress (NAEP), is a nationally representative and 
continuing assessment of what America’s students know and can 
do in various subject areas. Since 1969, assessments have been 
conducted periodically in reading, mathematics, science, writing, 
history, geography, and other fields. 

By making objective information on student performance 
available to policymakers at the national, state, and local levels, 
NAEP is an integral part of our nation’s evaluation of the con- 
dition and progress of education. Only information related to 
academic achievement is collected under this program. NAEP 
guarantees the privacy of individual students, their families, and 
their schools. 

NAEP is a congressionally mandated project of the National 
Center for Education Statistics within the Institute of 
Education Sciences of the U.S. Department of Education. The 



Commissioner of Education Statistics is responsible, by law, for 
carrying out the NAEP project through competitive awards to 
qualified organizations. 

In 1988, Congress established the National Assessment 
Governing Board (NAGB) to oversee and set policy for NAEP 
The Board is responsible for selecting the subject areas to be 
assessed; setting appropriate student achievement levels; develop- 
ing assessment objectives and test specifications; developing a 
process for the review of the assessment; designing the assessment 
methodology; developing guidelines for reporting and dis- 
seminating NAEP results; developing standards and procedures 
for interstate, regional, and national comparisons; determining 
the appropriateness of all assessment items and ensuring the 
assessment items are free from bias and are secular, neutral, and 
non-ideological; taking actions to improve the form, content, 
use, and reporting of results of the National Assessment; and 
planning and executing the initial public release of NAEP reports. 
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AMERICA’S CHARTER SCHOOLS 




Executive Summary 

The National Assessment of Educational Progress (NAEP) 
conducted a pilot study of America’s charter schools and 
their students as part of the 2003 NAEP assessments in 
reading and mathematics at the fourth-grade level. NAEP 
also surveyed participating charter schools about their prac- 
tices, structure, and governance. 

Charter schools are public schools of choice. A number of 
states have few or no charter schools; many charter schools 
have just recently opened; and some charter schools last 
only a few years. All of these factors make the selection of a 
representative sample challenging. 

For example, the final sample of 150 charter schools was 
obtainable only after multiple sources of information were 
consulted. Information from local school administrators, 
follow-up interviews, and field staff were used to update 
and verify the original school questionnaire data. 

While charter schools are similar to other public schools 
in many respects, they differ in several important ways, 
including the makeup of the student population and their 
location. For example, in comparison to other public 
schools, higher percentages of charter school fourth-grade 
students are Black and attend schools in central cities. 



Thus, when comparing the performance of charter and 
other public school students, it is important to compare 
students who share a common characteristic. For example, 
in mathematics, fourth-grade charter school students as a 
whole did not perform as well as their public school coun- 
terparts. However, the mathematics performance of White, 
Black, and Hispanic fourth- 
graders in charter schools 
was not measurably different 
from the performance of 
fourth-graders with similar 
racial/ ethnic backgrounds in 
other public schools. 

In reading, there was no 
measurable difference in 
performance between charter 
school students in the fourth 
grade and their public school 
counterparts as a whole. This 
was true even though, on average, charter schools have higher 
proportions of students from groups that typically perform 
lower on NAEP than other public schools have. In reading, 

as in mathematics, the 
performance of fourth- 
grade students with 
similar racial/ethnic 
backgrounds in char- 
ter schools and other 
public schools was not 
measurably different. 

There are also instances where the performance of students 
with shared characteristics differed. For example, among stu- 
dents eligible for free or reduced-price lunch, fourth-graders 
in charter schools did not score as high in reading or math- 
ematics, on average, as fourth-graders in other public schools. 

When considering these data, it should be noted that the 
charter school population is rapidly changing and growing. 
Future NAEP assessments may reveal different patterns of 
performance. Further, NAEP does not collect information 
about students’ prior educational experience, which contrib- 
utes to present performance. Nonetheless, the data in this 
report do provide a snapshot of charter school students’ cur- 
rent performance. 




. . . lower overall charter 
school mathematics 
performance, but no 
measurable differences 
among students with 
similar racial! ethnic 
backgrounds 




...no measurable 
difference in overall 
reading performance 
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Charter School Pilot Study 

As the charter school movement has grown, interest in how charter schools function 
and how their students perform academically has increased. Motivated by this interest, 
the National Assessment Governing Board (NAGB), which sets policy for the National 
Assessment of Educational Progress (NAEP), asked the National Center for Education 
Statistics (NCES) to conduct a pilot study of charter schools. This pilot study was con- 
ducted as part of NAEP s 2003 national assessment of fourth-graders in reading and 
mathematics. 

NAEP, as the nation’s report card, has a responsibility to gauge student progress in 
Americas schools. As a new kind of public school, charter schools are an appropriate sub- 
ject of study. The varied and changing nature of the charter school movement, however, 
makes such a study a challenge. 

This report first describes the pilot study’s design and methodology, within the context 
of a few lessons learned. Some key results are then presented separately for reading and 
mathematics in the body of the report, while other data are found in the appendix. As 
indicated in the appendix tables, some of the data presented in the appendix should be 
interpreted with caution due to the uncertainty of the estimates. Further explanation is 
provided in the Technical and Data Appendix at the end of this report. 



Who Attends Charter Schools? 



Grade 4: 2003 




Eligible for free/ 
reduced-price 
school lunch 




Students with 8* 

disabilities I I 11 

Limited-English- 9 

proficient students I 1 9 



Type of location 

Central city 

Non-central-city 
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Charter schools are public schools 
of choice. They serve as alternatives 
to the regular public schools to 
which students are assigned. While 
there are many similarities between 
charter and other public schools, 
they do differ in some important 
ways-including the makeup of 
the student population and their 
location, as shown in the graphic. 



Charter school students 
Other public school students 



•Significantly different from other public schools 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2003 Mathematics Charter School Pilot Study. 
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© LESSONS LEARNED 



► Until America’s charter 
schools become a more 
stable entity among educa- 
tional institutions, multiple 
sources of information and 
verification are necessary to 
accurately identify a repre- 
sentative sample of charter 
schools. 

► The unique characteristics 
of charter schools require 
additional information to be 
collected, beyond the infor- 
mation obtained from the 
regular NAEP questionnaires. 

► Since far more charter 
schools than other public 
schools are located in 
central cities, and there are 
striking differences in their 
demographic makeup, it 

is more fair to compare the 
performance of students 
that share a common 
characteristic. 

► Performance results vary for 
charter schools with different 
characteristics. There are 
wide variations in how char- 
ter schools are structured, 
the degree of oversight exert- 
ed by external agencies, and 
the length of time chartered. 






AMERICA’S CHARTER SCHOOLS 



Charter school students took the NAEP reading and mathematics assessments at the 
same time as students in all other schools. After the analyses for the main assessment were 
completed, different and sometimes conflicting sources of information were examined 
to confirm which sampled schools were, in fact, charter schools. Achievement results for 
students in charter schools were produced in the same manner as results for students in 
other public schools. 

Additional procedures were followed to make sure the sample of charter school 
students was large enough to conduct the necessary analyses. First, charter schools 
were selected within their state or jurisdiction proportional to their representation in the 
total population of charter schools. Second, charter schools were oversampled in three 
states — California, Michigan, and Texas — that together accounted for almost half of all 
charter school students nationally. 

There were a number of sources used to construct the final sample of charter schools. 
Initially, the 2000-2001 Common Core of Data, 1 updated by state departments of 
education, was used to sample charter schools. Then, the NAEP state coordinators inde- 
pendently verified the charter status of these schools. Additional charter schools were 
identified from the NAEP school questionnaire. Finally, in telephone interviews, a few 
schools were found not to be charter schools or not to have fourth-grade students eligible 
for the survey. A total of 150 schools were ultimately identified as charter schools, includ- 
ing 12 additional schools not originally identified on the NAEP website at the time of the 
2003 NAEP data release. These schools, most of which did not return a school question- 
naire, were discovered through the multiple sources of information just described. The 
results based on the full sample as well as results based on responses to the NAEP school 
questionnaire may be accessed on the NAEP data tool through the report's Web home 
page (http:/ / nces.ed.gov/ nationsreportcard/ studies/ charter/) . 




For More Info. 



More information 
about the NAEP 
Charter School Pilot 
Study can be found at 
http://nces.ed.gov/ 
nationsreportcard/ 
studies/charter/ . 

The NAEP website 
(http://nces.ed.gov/ 
nationsreportcard/) 
provides an array of 
information and results 
from both the main 
2003 assessments and 
the charter school pilot 
study, including PDF 
versions of all NAEP 
reports, a data tool for 
exploring the summary 
results, and a tool for 
examining released 
questions from the 
assessment. 



Within each of the 1 50 participating charter schools, a random sample of students partici- 
pated in either the reading or mathematics assessment — about half participated in reading 

and about half participated in mathematics. Table 1 displays the numbers of charter school students sampled for the pilot study 
as well as the numbers of other public school students sampled for the regular reading and mathematics assessments. 



Table 1 . Student sample size, by type of public school and subject assessed, grade 4: 2003 



Student sample size 



Subject 


In charter schools 


In other public schools 


Reading 


3,296 


188,148 


Mathematics 


3,238 


188,201 



SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, 
National Assessment of Educational Progress (NAEP), 2003 Reading and Mathematics Charter School Pilot Study. 



The Common Core of Data (CCD) is a program of the National Center for Education Statistics that annually compiles information about the nation’s public schools and school districts, 



and makes this information available through a public database. For more information, see http://nces.ed.gov/ccd/ . 
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Reading Results 



The average reading scores are reported 
on a 0-500 scale and are presented 
in figure 1 for fourth-grade students 
in charter schools and other public 
schools. In addition to the results for 
all students, results are presented by 
gender, race/ethnicity, eligibility for 
free/reduced-price school lunch, and 
type of school location. Note that 
results are not presented for students 
from race/ethnicity groups other than 
White, Black, and Hispanic, or for 
students for whom no information is 
available regarding their eligibility for 
free or reduced-price school lunch. 



It is important to note that simple 
cross-tabulations of a variable with 
measures of educational achievement, 
like the ones presented in this report, 
cannot constitute proof that a differ- 
ence in the variable causes differences 
in educational achievement. There are 
many reasons why the performance 
of one group of students differs from 
another. 

There was no measurable difference 
between the reading scores of charter 
school students and other public school 
students overall, although there were 
some differences in results for certain 



groups of students. Female students 
in charter schools scored lower, on 
average, than female students in other 
public schools. Also, charter school 
students eligible for free/ reduced-price 
school lunch scored lower than eligible 
students in other public schools. 

On the other hand, the reading 
scores for White, Black, and Hispanic 
students in charter schools were not 
measurably different from those for 
students with the same racial/ ethnic 
background in other public schools. 
There were also no measurable differ- 
ences between average reading scores 



Figure 1 . Average scale score in reading for charter and other public school students overall, and by student and school characteristics, grade 4: 2003 



Scale score 




All students Gender Race/ ethnicity Eligibility for free/ reduced-price Type of location 



school lunch 

| Charter school students Other public school students 



‘Significantly different from other public schools. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 Reading Charter 
School Pilot Study. 



What are NAEP Achievement Levels? 



Achievement levels are performance standards set by the National Assessment Governing Board (NAGB) to 
help interpret student performance on NAER The three NAEP achievement levels, from lowest to highest, are 

Bas/c-denotes partial mastery of the knowledge and skills that are 
fundamental for proficient work at a given grade. 

Proficient - represents solid academic performance. Students reaching this level have 
demonstrated competency over challenging subject matter. 

Advanced-signifies superior performance. 
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AMERICA’S CHARTER SCHOOLS 



of fourth-grade students in charter 
schools and other public schools in 
central cities or in non-central-city 
locations. 

As shown in table 2, the percent- 
ages of charter school students 
performing at or above Basic and at 
or above Proficient in reading were 
not measurably different from the 
percentages of other public school 
students when comparing students 
overall. 

The score differences seen between 
some groups of fourth-grade stu- 
dents in charter schools and other 
public schools were not evident in 
the achievement-level results. The 
observed differences in percentages at 
or above Basic and Proficient achieve- 
ment levels were not significant for 
any groups defined by gender, race/ 
ethnicity, eligibility for free/reduced- 
price lunch, or type of school 
location. 



Table 2. Percentage of students at or above Basic and Proficient in reading for charter and other 
public school students overall, and by student and school characteristics, grade 4: 2003 



Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


All students 


58 


62 


27 


30 


Gender 










Male 


55 


58 


24 


26 


Female 


60 


65 


29 


33 


Race/ethnicity 










White 


73 


74 


39 


39 


Black 


37 


39 


12 


12 


Hispanic 


45 


43 


17 


14 


Eligibility for free/ reduced-price school lunch 










Eligible 


39 


45 


12 


15 


Not eligible 


72 


76 


38 


41 


Type of location 










Central city 


50 


51 


21 


22 


Non-central-city 


66 


66 


33 


33 



SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP), 2003 Reading Charter School Pilot Study. 




Detailed descriptions of the NAEP reading and mathematics achievement levels can be found on 
the NAGB website ( http://www.nagb.org/pubs/pubs.html ). 

As provided by law, NCES, upon review of congressionally mandated evaluations of NAEP has 
determined that achievement levels are to be used on a trial basis and should be interpreted and 
used with caution. Additional information about the trial status of achievement levels is available on 
the NAEP website ( http://nces.ed.gov/nationsreportcard/achlevdev.asp ). 
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Table 3 shows the percentage of stu- 
dents assessed, average reading score, 
and achievement-level results for char- 
ter school students and other public 
school students who were taught by 
teachers who differed in type of teach- 
ing certificate and years of experience. 
Proportionally more fourth-graders in 
charter schools were taught by teachers 
without regular certification (e.g., with 
probationary, provisional, temporary, or 



emergency certification) and by teachers 
with four years of teaching experience 
or less. One limitation is that teachers 
were not asked whether they had alter- 
native teacher certification, a route now 
available in many states and districts 
for teachers in charter and other public 
schools. 

There was no measurable difference 
in the performance of charter school 
and other public school fourth-grade 



students taught by teachers with regular 
or other forms of certification. 

Among fourth-graders with less 
experienced teachers, charter school stu- 
dents had a lower average reading score 
than other public school students. The 
observed differences in reading per- 
formance between students in charter 
schools and other public schools who 
had more experienced teachers were not 
statistically significant. 



Table 3. Percentage of students assessed, average scale score, and achievement-level results in reading, by type of public school and teacher 
characteristics, grade 4: 2003 





Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


Type of teaching certificate 


















Regular 


69* 


90 


215 


218 


60 


63 


28 


31 


Other 1 


22* 


10 


207 


209 


51 


53 


23 


24 


Years of teaching experience 


















0 to 4 years 


42* 


23 


205* 


213 


48* 


57 


20 


26 


5 to 9 years 


29* 


22 


213 


215 


58 


61 


26 


29 


10 years or more 


29* 


55 


221 


219 


68 


65 


35 


32 



•Significantly different from other public schools. 

ln 0ther" includes the following types of teaching certificate: probationary, provisional, temporary, and emergency. Results are not shown for students whose teachers reported not 
having a certificate of any kind. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

2003 Reading Charter School Pilot Study. 



Table 4 shows that 55 percent of 
charter school students in the fourth 
grade attended a school that was a 
part of a school district with other 
public schools. The remaining stu- 
dents attended a charter school that 
was a school district by itself. 

The average reading score for 
students in charter schools that 
were a part of a larger public school 
district was higher than the score 
for students in charter schools that 



Table 4. Percentage of students assessed, average scale score, and achievement-level results in reading, 
by type of charter school governance, grade 4: 2003 



Type of governance 


Percent of 
students 
assessed 


Average scale 
score 


Percent at or 
above 
Basic 


Percent at or 
above 
Proficient 


Other public schools 


100 


217 


62 


30 


Charter schools 

Part of a public school district 


55 


218 


64 


31 


Charter school district by itself 


45 


208* 


52* 


23* 



’Significantly different from other public schools. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2003 Reading Charter School Pilot Study. 



were their own district, but was not 
measurably different from that of 
students in other public schools. 
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Mathematics Results 

The average mathematics scores are 
reported on a 0-500 scale, but because 
different dimensions define the frame- 
works for each subject, the mathematics 
results cannot be compared to those in 
reading. 

Figure 2 shows the average math- 
ematics scores for fourth-grade students 
in charter schools and other public 
schools overall, as well as by gender, 
race/ ethnicity, eligibility for free/ 
reduced-price school lunch, and type of 
school location. Recall that results are 
not presented for students from some 
race/ ethnicity groups or for students 
for whom no information is available 



regarding their eligibility for free or 
reduced-price school lunch. 

The national results showed a lower 
average mathematics score overall 
for fourth-grade students in charter 
schools. This was true for both male 
and female students. However, there 
were no measurable differences when 
comparisons were made for fourth- 
graders with similar racial/ ethnic 
backgrounds. 

The average score for fourth-grade 
charter school students who were eli- 
gible for free or reduced-price school 
lunch was lower than that of their 



peers in other public schools. In 
contrast, there was no measurable 
difference in the scores for stu- 
dents who were not eligible. 

Fourth-grade students who 
attended charter schools in cen- 
tral cities scored lower on average 
than fourth-grade students who 
attended other public schools in 
similar locations. There was no 
measurable difference, however, 
in the average mathematics score 
between charter school and other 
public school students in non- 
central-city locations. 



Figure 2. Average scale score in mathematics for charter and other public school students overall, and by student and school characteristics, 
grade 4: 2003 



White 

242 243 




All students 



Gender 



Not eligible 

244 

238 



Hispanic 

219 221 



Eligible 

222 

216 



Non-central-city 

Central city 236 237 

227 

221 * 



Race/ethnicity 



Eligibility for free/reduced-price 
school lunch 



Type of location 

| Charter school students Q Other public school students 



^Significantly different from other public schools. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 Mathematics 
Charter School Pilot Study. 
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The mathematics achievement-level 
results presented in table 5 reflect the 
same general pattern observed in aver- 
age mathematics scores for students 
attending charter and other public 
schools. The percentages of fourth- 
grade students at or above Basic and at 
or above Proficient were lower in charter 
schools than in other public schools for 
students overall. 

When achievement-level results are 
examined by student groups, some are 
significantly different and some are not. 



For example, the percentages of male 
and female fourth-graders in charter 
schools at or above Basic are lower than 
those for other public schools, but the 
percentages at or above Proficient are 
not measurably different. 

There were no measurable differences 
in results by achievement levels when 
comparisons were made for fourth- 
grade students in charter schools and 
other public schools with similar racial/ 
ethnic backgrounds. 



The percentages of charter school 
fourth-graders who are and are not eli- 
gible for free or reduced-price lunch at 
or above the Basic level are lower than 
those for other public schools, but the 
percentages at or above Proficient are 
not measurably different. 

For schools in central cities, the 
percentages of charter school fourth- 
graders performing at or above the Basic 
and Proficient levels were lower than 
the percentages of other public school 
fourth-graders. 



Table 5. Percentage of students at or above Basic and Proficient in mathematics for charter and other 
public school students overall, and by student and school characteristics, grade 4: 2003 



Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


All students 


69* 


76 


25* 


31 


Gender 










Male 


69* 


77 


27 


34 


Female 


68* 


75 


24 


29 


Race/ethnicity 










White 


84 


87 


42 


42 


Black 


51 


54 


9 


10 


Hispanic 


58 


62 


12 


15 


Eligibility for free/reduced-price school lunch 










Eligible 


53* 


62 


12 


15 


Not eligible 


81* 


88 


37 


45 


Type of location 










Central city 


* 

00 

LD 


67 


16* 


23 


Non-central-city 


79 


80 


34 


35 



’Significantly different from other public schools. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National 
Assessment of Educational Progress (NAEP), 2003 Mathematics Charter School Pilot Study. 



CD 


For More Info... 


I 


Explanations of how 
school and student 
variables are collected 
and analyzed are 




available on the NAEP 


j 


website (htto://nces. 


ill 


ed.gov/nationsreDort- 


m 

i 


card/mathematics/ 
resu Its2003/interoret- 
results.asD#ReoGrouDs). 
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Table 6 shows the percentage of stu- 
dents assessed, average mathematics 
scores, and achievement-level results 
for charter and other public school 
students who had teachers who dif- 
fered in type of teaching certification 
and years of teaching experience. As 
seen in reading, higher percentages of 
fourth-graders in charter schools were 



taught by teachers without regular 
certification (e.g., with probationary, 
provisional, temporary, or emergency 
certification) and by teachers with four 
years of teaching experience or less. 
Among fourth-graders whose teachers 
had taught for four years or less, charter 
school students scored lower on aver- 
age in mathematics than other public 



school students. There was no measur- 
able difference in the performance of 
fourth-grade students in charter schools 
and fourth-grade students in other 
public schools when taught by teachers 
with five or more years of experience, 
with a regular teaching certificate, or 
with another type of certificate. 



Table 6. Percentage of students assessed, average scale score, and achievement-level results in mathematics, by type of public school and 
teacher characteristics, grade 4: 2003 





Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


Charter 

schools 


Other public 
schools 


Type of teaching certificate 


















Regular 


66* 


90 


230 


235 


71 


78 


28 


32 


Other 1 


22* 


10 


227 


228 


67 


68 


22 


26 


Years of teaching experience 


















0 to 4 years 


43* 


23 


223* 


231 


61* 


73 


19* 


28 


5 to 9 years 


29* 


22 


230 


233 


72 


75 


28 


31 


10 years or more 


27* 


54 


233 


236 


74 


79 


31 


34 



’Significantly different from other public schools. 

^'Other" includes the following types of teaching certificate: probationary, provisional, temporary, and emergency. Results are not shown for students whose teachers reported not 
having a certificate of any kind. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

2003 Mathematics Charter School Pilot Study. 



g 



Table 7 shows that about half 
the fourth-grade students in char- 
ter schools attended a school that 
was part of a larger public school 
district and about half attended a 
school that was a district by itself. 
As in reading, the average math- 
ematics score for fourth-grade 
students in charter schools that 
were part of a public school dis- 
trict was higher than the score for 
students in charter schools that 
were their own district, and was 
not measurably different from the 
score for students in other public 
schools. 



Table 7. Percentage of students assessed, average scale score, and achievement-level results in mathematics, 
by type of charter school governance, grade 4: 2003 



Type of governance 


Percent 
of students 
assessed 


Average scale 
score 


Percent at or 
above 
Basic 


Percent at or 
above 
Proficient 


Other public schools 


100 


234 


76 


31 


Charter schools 

Part of a public school district 


52 


234 


75 


32 


Charter school district by itself 


48 


225* 


66* 


20* 



’Significantly different from other public schools. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of 
Educational Progress (NAEP), 2003 Mathematics Charter School Pilot Study. 
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MATHEMATICS 




Conclusions 

The nature of the NAEP survey design allows for only a 
snapshot of America’s charter schools. And, given the con- 
siderable variation in student and school characteristics, that 
picture is not so easily discerned. For example, for students 
from the same racial/ethnic backgrounds, reading and math- 
ematics performance in charter schools did not differ from 
that in other public schools. However, this study found lower 
overall mathematics performance in charter schools than in 
other public schools. 

On the other hand, in reading there was no measurable 
difference between the overall performance of charter school 
fourth-grade students as a whole and their counterparts in 
other public schools. This is true even though, on average, 
charter schools have higher proportions of students from 
groups that typically perform lower on NAEP than other 
public schools, such as minority students and students in cen- 
tral cities. Such patterns illustrate how important it is to look 
beyond simple comparisons of the two school types. 



NCES will continue its analyses of the 2003 pilot charter 
school data, using more advanced statistical analysis methods 
to portray complex relationships related to student perfor- 
mance in charter and other public schools. 

This study has provided important lessons for assessing 
charter schools. As a result, NAEP has improved its proce- 
dures for identifying charter schools. 

In 2005, NAEP will assess three subjects — reading, 
mathematics, and science — in fourth and eighth grades at the 
state level, and will assess more students than ever before. A 
side benefit of the large assessment is that the NAEP sample 
will include a representative sample of charter schools with- 
out oversampling. NAEP will continue to identify charter 
schools as a separate category in the data placed on the NAEP 
website (http:/ / nces.ed.gov/ nationsreportcard) at the time of 
the initial release of each assessment, providing educators, 
researchers, policymakers, and the public with data 
with which to measure the educational progress of Americas 
students. 
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AMERICA’S CHARTER SCHOOLS 



Technical and Data Appendix 



NAEP Sampling Procedures 

The schools and students participating in NAEP assessments 
are chosen to be nationally representative. Samples of schools 
and students are drawn from each state and from certain 
other participating jurisdictions (for example, the District of 
Columbia). The results from the assessed students are com- 
bined and weighted to provide accurate estimates of overall 
national performance as well as of the performance of indi- 
vidual states and other jurisdictions. The weighting procedure 
is used to take into account the fact that states, and schools 
within states, represent different proportions of the overall 
national population. Since samples in most states are roughly 
the same (to allow for stable state estimates and administra- 
tive efficiencies), the results for students in less populous 
states are assigned smaller sampling weights than the results 
for students in more populous states. The sampling weights 
are also used in estimating percentages of students in various 
subgroups. 

School and Student Participation Rates 

The school participation rate for charter schools was 100 
percent for both the reading and mathematics assessments 
(which were conducted in the same schools). The school par- 
ticipation rates for other public schools in both assessments 
were 100 percent (6,764 schools participating). The weighted 
student participation rate was 92 percent for charter schools 
in both the reading and mathematics assessments. For other 
public schools the weighted student participation rate was 
94 percent in both assessments. These rates were well within 
the NCES standards for ensuring unbiased samples and 
reporting data. 



Assessed Students 

Every effort is made to ensure that all sampled students 
who are capable of participating in the assessment are 
assessed. A sampled student who is identified by the school 
as a student with a disability or as a limited-English- 
proficient student may be tested with accommodations 
allowed by NAEP; students so identified may be excluded 
from the assessment if they do not meet criteria for inclusion 
established by NAEP. The numbers of students assessed in 
each subject varied somewhat because more students tend to 
be excluded from reading assessments than mathematics 
assessments. In 2003, the exclusion rates for reading were 
4 percent in charter schools and 6 percent in other public 
schools, and the rates for mathematics were 2 percent and 
4 percent, respectively. 

Definitions of Variables 

Student race/ ethnicity is presented for White, Black, 

Flispanic, or “Other” racial/ethnic groups based on school 
records. The “Other” group includes Asian/Pacific Islander 
students, American Indian/Alaska Native students, and 
students categorized in school records as another race or eth- 
nicity. The results for the “Other” group are not presented in 
the body of the report because sample sizes are small but are 
included in the appendix. School location is categorized as 
“central city” or “non-central-city” based on information from 
the U.S. Census Bureau. The term “central city” does not 
refer to “inner city,” but to the central part of a metropolitan 
statistical area. Non-central-city locations include urban fring- 
es of metropolitan statistical areas, large towns, small towns, 
and rural areas. 
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TECHNICAL 

INFORMATION 




THE NATION’S REPORT CARD 



Interpreting Statistical Significance 

The differences between statistics — such as students’ average 
scale scores and percentages of students — that are discussed in 
this report are determined by using statistical measures known 
as standard errors. Standard errors for the NAEP scores and 
percentages presented in this report are available on the 
NAEP web site (http:// nces.ed.gov/ nationsreportcard/ naep- 
data/) . Comparisons are based on statistical tests that consider 
both the size of the differences and the standard errors of the 
two statistics being compared. Estimates based on smaller 
subgroups are likely to have relatively large standard errors. As 
a consequence, a numerical difference that seems large may 
not be statistically significant. When this is the case, the term 
“no measurable difference” is used in the report. Furthermore, 
differences of the same magnitude may or may not be sta- 
tistically significant depending upon the size of the standard 
errors of the statistics. For example, a 5 -point difference 
between male and female students may be statistically sig- 
nificant, while a 5-point difference between White and Black 
students may not be. The differences described in this report 
have been determined to be statistically significant at the .05 
level with appropriate adjustments for multiple comparisons. 



In the tables and charts of this report, the symbol (*) is used 
to indicate that a charter school score or percentage is signifi- 
cantly different from the comparable measure for other public 
schools. The symbol (!) is used to indicate that the nature of 
the sample — its size or its variance — does not allow accurate 
determination of the variability of the statistics and that any 
significance tests involving these statistics should be interpret- 
ed cautiously. 

Statistically significant differences among groups of charter 
school students — for example, between White students and 
Black students — are not identified in the table and charts, 
but they were tested in the same way. Any difference between 
scores or percentages that is identified as higher, lower, larger, 
or smaller in this report, including within-group differences 
not marked in tables and charts, meets the requirements for 
statistical significance. 
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AMERICA’S CHARTER SCHOOLS 



Reading Data 



Table A-l. Percentage of students assessed, average scale score, and achievement-level results in reading, by type of public school and 
student characteristics, grade 4: 2003 



Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Gender 


















Male 


51 


51 


210 


213 


55 


58 


24 


26 


Female 


49 


49 


215* 


220 


60 


65 


29 


33 


Race/ethnicity 


















White 


49 


59 


227 


227 


73 


74 


39 


39 


Black 


29* 


17 


195 


197 


37 


39 


12 


12 


Hispanic 


18 


18 


201 


199 


45 


43 


17 


14 


Other 1 


4 * 


6 


219 


219 


68 


64 


33 


32 


Eligibility for free/ reduced-price 
school lunch 


















Eligible 


41 


44 


195* 


201 


39 


45 


12 


15 


Not eligible 


50 


52 


226 


229 


72 


76 


38 


41 


Information not available 


9 


4 


218! 


219 


63! 


65 


32! 


33 


Students with disabilities 


















Yes 


7 


10 


180 


184 


26 


29 


7 


9 


No 


93 


90 


215* 


220 


60 


65 


28 


32 


Limited-English-proficient students 


















Yes 


8 


8 


183 


186 


26 


28 


4 


7 


No 


92 


92 


215 


219 


60 


65 


29 


32 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

‘Significantly different from other public schools. 

1 "0ther" includes Asian/Pacific Islander students, American Indian/Alaska Native students, and students categorized in school records as another race or ethnicity. 

NOTE: The results for students with disabilities and limited-English-proficient students are based on students who were assessed and cannot be generalized to the total population 
of such students, which also includes students who could not be assessed based on defined exclusion criteria. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 

2003 Reading Charter School Pilot Study. 
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READING DATA 




THE NATION’S REPORT CARD 



Table A-2. Percentage of students assessed, average scale score, and achievement-level results in reading, by type of public school and 
school characteristics, grade 4: 2003 



Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Region of the country 


















Northeast 


8* 


18 


t 


223 


t 


69 


4 


36 


Midwest 


24 


22 


205* 


220 


49* 


66 


21* 


33 


South 


33 


36 


215! 


215 


61! 


60 


29! 


28 


West 


35* 


24 


214 


210 


59 


55 


29 


25 


Type of location 


















Central city 


51* 


29 


205 


208 


50 


51 


21 


22 


Non-central-city 1 


49* 


71 


220 


220 


66 


66 


33 


33 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

^Reporting standards not met. Sample size is insufficient to permit a reliable estimate. 

‘Significantly different from other public schools. 

fNon-central-city includes students attending schools in urban fringe/large town and rural/small town locations. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2003 Reading Charter School Pilot Study. 



Table A-3. Percentage of students assessed, average scale score, and achievement-level results in reading, by type of public school and 
teacher characteristics, grade 4: 2003 



Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Type of teaching certificate 


















Regular 


69* 


90 


215 


218 


60 


63 


28 


31 


Other 1 


22* 


10 


207 


209 


51 


53 


23 


24 


None 


9* 


# 


200! 


t 


44! 


t 


17! 


t 


Years of teaching experience 


















0 to 4 years 


42* 


23 


205* 


213 


48* 


57 


20 


26 


5 to 9 years 


29* 


22 


213 


215 


58 


61 


26 


29 


10 years or more 


29* 


55 


221 


219 


68 


65 


35 


32 



#The estimate rounds to zero. 

! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

^Reporting standards not met. Sample size is insufficient to permit a reliable estimate. 

‘Significantly different from other public schools. 

^‘Other” includes the following types of teaching certificate: probationary, provisional, temporary and emergency. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2003 Reading Charter School Pilot Study. 
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AMERICA’S CHARTER SCHOOLS 



Table A-4. Percentage of students assessed, average scale score, and achievement-level results in reading, by charter school characteristics, 
grade 4: 2003 



Characteristic 


Percent of students 
assessed 


Average 
scale score 


Percent 
at or above 
Basic 


Percent 
at or above 
Proficient 


Other public schools 


100 


217 


62 


30 


Charter schools 

Newly created charter school or pre-existing school 










Newly created school 


70 


213 


58 


27 


Pre-existing school 


30 


211 


56 


25 


Charter-granting agency 










School district 


49 


218 


65 


32 


State board of education 


27! 


208!* 


52! 


23! 


Postsecondary institution 


15 


203* 


45* 


19* 


State charter-granting agency 


6! 


214! 


60! 


29! 


Other 


1! 


197!* 


421* 


14!* 


Years since the school’s charter was granted 










0 to 1 year 


7! 


223! 


70! 


34! 


2 to 3 years 


31 


208 


53 


23 


4 to 5 years 


29 


212 


57 


26 


6 years or more 


33! 


215! 


60! 


29! 


Years providing instruction as a charter school 










0 to 1 year 


8! 


225!* 


721* 


34! 


2 to 3 years 


32! 


214! 


60! 


28! 


4 to 5 years 


37 


212 


56 


26 


6 years or more 


23 


210 


54 


25 


Type of student population served 










General student population 


92 


214 


60 


28 


Special student target population 1 


8 


194!* 


36!* 


141* 


Type of governance 










Part of a public school district 


55 


218 


64 


31 


Charter school district by itself 


45 


208* 


52* 


23* 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

’Significantly different from other public schools. 

iThe special student target population includes at-risk students, students with disabilities, gifted/talented students, and other students. 

NOTE: Detail may not sum to totals because of rounding or because the "don't know" category was excluded from the table. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2003 Reading Charter School Pilot Study. 
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READING DATA 




THE NATION’S REPORT CARD 



Table A-5. Percentage of students assessed, average scale score, and achievement-level results in reading, by waivers or exemptions from state 
or district policies, grade 4: 2003 



Waivers 


Percent of students 
assessed 


Average 
scale score 


Percent 
at or above 
Basic 


Percent 
at or above 
Proficient 


Other public schools 


100 


217 


62 


30 


Charter schools 

Teacher certification requirements 










Yes 


23! 


214! 


59! 


30! 


No 


77 


212 


57 


26 


Teacher/staff hiring/firing policies 










Yes 


30 


214 


59 


29 


No 


68 


211 


56 


26 


Curriculum requirements 










Yes 


37 


223* 


70* 


36 


No 


60 


205* 


49* 


21* 


Control of finances/budget 










Yes 


33! 


221! 


67! 


33! 


No 


63 


209* 


53* 


24* 


Incentives, rewards, or sanctions due to school 
performance 










Yes 


17! 


221! 


67! 


36! 


No 


81 


211 


56 


25 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

•Significantly different from other public schools. 

NOTE: Detail may not sum to totals because of rounding or because the "don't know" category was excluded from the table. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2003 Reading Charter School Pilot Study. 
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AMERICA’S CHARTER SCHOOLS 



Mathematics Data 



Table A-6. Percentage of students assessed, average scale score, and achievement-level results in mathematics, by type of public school and 
student characteristics, grade 4: 2003 





Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Characteristic 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Gender 


















Male 


48* 


51 


229* 


235 


69* 


77 


27 


34 


Female 


52* 


49 


228* 


233 


68 * 


75 


24 


29 


Race/ethnicity 


















White 


45* 


58 


242 


243 


84 


87 


42 


42 


Black 


31* 


17 


214 


216 


51 


54 


9 


10 


Hispanic 


20 


19 


219 


221 


58 


62 


12 


15 


Other 1 


4 * 


6 


235 


241 


76 


82 


33 


41 


Eligibility for free/reduced-price 
school lunch 


















Eligible 


42 


44 


216* 


222 


53* 


62 


12 


15 


Not eligible 


48 


52 


238 


244 


81* 


88 


37 


45 


Information not available 


10 


4 


232! 


236 


74! 


77 


28! 


34 


Students with disabilities 


















Yes 


8 * 


11 


209 


214 


42 


50 


8 


13 


No 


92* 


89 


230* 


237 


71* 


80 


27 


34 


Limited-English-proficient students 


















Yes 


9 


9 


211 


214 


45 


49 


5 


9 


No 


91 


91 


230* 


236 


71* 


79 


27 


34 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

*Significantly different from other public schools. 

^Other" includes Asian/Pacific Islander students, American Indian/Alaska Native students, and students categorized in school records as another race or ethnicity. 

NOTE: The results for students with disabilities and limited-English-proficient students are based on students who were assessed and cannot be generalized to the total population of 
such students, which also includes students who could not be assessed based on defined exclusion criteria. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 
Mathematics Charter School Pilot Study. 
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MATHEMATICS 

DATA 




THE NATION’S REPORT CARD 



Table A-7. Percentage of students assessed, average scale score, and achievement-level results in mathematics, by type of public school and 
school characteristics, grade 4: 2003 



Characteristic 


Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Region of the country 


















Northeast 


10* 


17 


t 


238 


t 


80 


t 


36 


Midwest 


23 


22 


221* 


237 


59* 


79 


17* 


35 


South 


30 


36 


231! 


233 


72! 


76 


30! 


29 


West 


37* 


24 


230 


230 


71 


71 


28 


27 


Type of location 


















Central city 


50* 


29 


221* 


227 


58* 


67 


16* 


23 


Non-central-city 1 


50* 


71 


236 


237 


79 


80 


34 


35 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

^Reporting standards not met. Sample size is insufficient to permit a reliable estimate. 

•Significantly different from other public schools. 

iNon-central-city includes students attending schools in urban fringe/large town and rural/small town locations. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2003 Mathematics Charter School Pilot Study. 



Table A-8. Percentage of students assessed, average scale score, and achievement-level results in mathematics, by type of public school 
and teacher characteristics, grade 4: 2003 



Characteristic 


Percent of students 
assessed 


Average scale score 


Percent at or above 
Basic 


Percent at or above 
Proficient 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Charter 

schools 


Other 

public 

schools 


Type of teaching certificate 


















Regular 


66* 


90 


230 


235 


71 


78 


28 


32 


Other 1 


22* 


10 


227 


228 


67 


68 


22 


26 


None 


12* 


# 


218 


t 


57 


t 


15 


t 


Years of teaching experience 


















0 to 4 years 


43* 


23 


223* 


231 


61* 


73 


19* 


28 


5 to 9 years 


29* 


22 


230 


233 


72 


75 


28 


31 


10 years or more 


27* 


54 


233 


236 


74 


79 


31 


34 



#The estimate rounds to zero. 

Mnterpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

^Reporting standards not met. Sample size is insufficient to permit a reliable estimate. 

•Significantly different from other public schools. 

1 “0ther” includes the following types of teaching certificate: probationary, provisional, temporary and emergency. 

NOTE: Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2003 
Mathematics Charter School Pilot Study. 
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AMERICA’S CHARTER SCHOOLS 



Table A-9. Percentage of students assessed, average scale score, and achievement-level results in mathematics, by charter school 
characteristics, grade 4: 2003 



Characteristic 


Percent 
of students 
assessed 


Average 
scale score 


Percent 
at or above 
Basic 


Percent 
at or above 
Proficient 


Other public schools 


100 


234 


76 


31 


Charter schools 

Newly created charter school or pre-existing school 










Newly created school 


68 


229 


69* 


26 


Pre-existing school 


32 


228!* 


68!* 


24!* 


Charter-granting agency 










School district 


45 


235 


77 


33 


State board of education 


30 


222!* 


61!* 


18!* 


Postsecondary institution 


15 


222* 


60* 


17* 


State charter-granting agency 


6 


225! 


65! 


24! 


Other 


2 


224! 


65! 


23! 


Years since the school’s charter was granted 










0 to 1 year 


8 


234! 


76! 


30! 


2 to 3 years 


28 


224 


63 


20 


4 to 5 years 


33 


226 


66 


22 


6 years or more 


31 


233! 


74! 


31! 


Years providing instruction as a charter school 










0 to 1 year 


10 


235! 


80! 


28! 


2 to 3 years 


27 


232! 


72! 


32! 


4 to 5 years 


40 


227 


67 


23 


6 years or more 


24 


228* 


69* 


23* 


Type of student population served 










General student population 


91 


230 


70* 


27 


Special student target population 1 


9 


219!* 


57!* 


12!* 


Type of governance 










Part of a public school district 


52 


234 


75 


32 


Charter school district by itself 


48 


225* 


66* 


20* 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

’Significantly different from other public schools. 

iThe special student target population includes at-risk students, students with disabilities, gifted/talented students, and other students. 

NOTE: Detail may not sum to totals because of rounding or because the "don't know" category was excluded from the table. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2003 Mathematics Charter School Pilot Study. 
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MATHEMATICS 

DATA 




THE NATION’S REPORT CARD 



Table A-10. Percentage of students assessed, average scale score, and achievement-level results in mathematics, by waivers or exemptions 
from state or district policies, grade 4: 2003 



Waivers 


Percent 
of students 
assessed 


Average 
scale score 


Percent 
at or above 
Basic 


Percent 
at or above 
Proficient 


Other public schools 


100 


234 


76 


31 


Charter schools 

Teacher certification requirements 










Yes 


21 


229 


69 


27 


No 


79 


228 


69* 


25 


Teacher/staff hiring/firing policies 










Yes 


28 


231 


73 


28 


No 


70 


227 


67 


24 


Curriculum requirements 










Yes 


35 


238 


79 


37 


No 


64 


223* 


63* 


19* 


Control of finances/budget 










Yes 


30 


236! 


78! 


34! 


No 


67 


225* 


65* 


22* 


Incentives, rewards, or sanctions due to school 
performance 










Yes 


16 


237! 


79! 


35! 


No 


81 


227 


67* 


23 



! Interpret data with caution. The nature of the sample does not allow accurate determination of the variability of the statistic. 

•Significantly different from other public schools. 

NOTE: Detail may not sum to totals because of rounding or because the "don't know" category was excluded from the table. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 
2003 Mathematics Charter School Pilot Study. 
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